Chapter 2. Sufficiency and Estimation

2.1 Sufficient statistics. Classically, a " statistic " is a measurable function of the observations , say f (X 1 ,. .. , X n). The concept of " statistic " differs from that of " random variable " in that a random variable is defined on a probability space, so that one probability measure is singled out, where for statistics we have in mind a family of possible probability measures, R. A. Fisher (1922) called a statistic sufficient " when no other statistic which can be calculated from the same sample provides any additional information as to the value of the parameter to be estimated. " For example, given a sample (X 1 ,. .. , X n) where the X j are i.i.d. N (θ, 1), the sample mean X := (X 1 + · · · X n)/n turns out to be a sufficient statistic for the unknown parameter θ. J. Neyman (1935) gave one form of a " factorization theorem " for sufficient statistics. This section will give more general forms of the definitions and theorem due to Halmos and L. J. Savage (1949). In general, given a measurable space (S, B), that is a set S with a σ-algebra B of subsets, and another measurable space (Y, F), a statistic is a measurable function T from S into Y. Often, Y = R or a Euclidean space R d with Borel σ-algebra. Let T −1 (F) := {T −1 (F) : F ∈ F}. Then T −1 (F) is a σ-algebra and is the smallest σ-algebra on S for which T is measurable. For any measure µ on B, L 1 (µ) := L 1 (S, B, µ) denotes the set of all real-valued, B-measurable, µ-integrable functions on S. For any probability measure P on (S, B), sub-σ-algebra A ⊂ B, and f ∈ L 1 (P), we have a conditional expectation E P (f |A), defined as a function g ∈ L 1 (S, A, P), so that g is measurable for A, such that for all A ∈ A, ∫ A gdP = ∫ A fdP. Such a g always exists, as a Radon-Nikodym derivative of the signed measure A → ∫ A f dP with respect to the restriction of P to A (RAP, Theorems 5.5.4, 10.1.1). If f = 1 B for some B ∈ B let P (B|A) := E …